Misuse detection is the process of detecting and reporting uses of processing systems and networks that would be deemed inappropriate or unauthorized if known to the responsible parties. Even though designers, owners, and administrators of systems and networks usually try to prevent misuses, the complexity of modern system environments and the difficulty of preventing authorized users from abusing their privileges make it virtually impossible to anticipate and prevent all possible security problems. To date, however, there is no known system or method for effectively and independently detecting and reporting misuses and facilitating their subsequent investigation.
The direct financial impact of computer misuse is very high and increasing. The National Institute of Justice (NIJ), for example, estimates the cost of computer misuse in the U.S. during 1993 to range from $500 million to $5 billion dollars. In addition, NIJ estimates that $2.1 billion was lost in the same period from telephone service fraud through illegally manipulating telephone company computer systems. In fact, virtually every sector of modern commerce and government, from banking to national defense, depends on the security of processing systems on which the sectors rely. As an increasing number of organizations connect their internal networks to outside public networks (e.g. the Internet, "National Information Infrastructure", etc.), the potential importance of misuse increases. This is because vulnerability increases with increased exposure.
Processing system misuse detection and reporting research has been funded by U.S. government agencies who have concerns for the confidentiality of their computer systems. Researchers have generally been associated with large research organizations or national laboratories. These institutions have required detailed knowledge of technical computer security, known threats and vulnerabilities, protection mechanisms, standard operational procedures, communications protocols, details of various systems' audit trails, and legal investigation of computer crimes. This misuse detection and reporting research has followed two basic approaches: anomaly detection systems and expert systems, with the overwhelming emphasis on anomaly detection.
Anomaly detection looks for statistically anomalous behavior. It assumes that intrusions and other security problems are rare and that they appear unusual when compared to other user behavior. D. Denning, "An Intrusion Detection Model," Proc 1986 IEEE Symp. Security & Privacy, (April 1986) provides an anomaly detection model (hereinafter the "Denning Model") for detecting intrusions into computer systems. The Denning Model uses statistical profiles for user, dataset, and program usage to detect "exceptional" use of the system.
There are variations of the Denning Model of anomaly detection models and different applications of these models. Anomaly detection techniques such as those based on the Denning Model, however, have generally proven to be ineffective and inefficient. Anomaly detection techniques, for instance, do not detect most actual misuses. The assumption that computer misuses would appear statistically anomalous has been proven false. When scripts of known attacks and misuses are replayed on computers with statistical anomaly detection systems, few if any of the scripts are identified as anomalous. This occurs because the small number of commands in these scripts are insufficient to violate profiling models.
In general, anomaly detection techniques can not detect particular instances of misuses unless the specific behaviors associated with those misuses also satisfy statistical tests without security relevance. Anomaly detection techniques also produce false alarms. Most of the reported anomalies are purely statistical and do not reflect security problems. These false alarms often cause system managers to resist using anomaly detection method because they increase the processing system workload without substantial benefits.
Another limitation with anomaly detection approaches is that users activities are often too varied for a single profile can result in many false alarms. Statistical measures also are not sensitive to the order in which events occur, and this may prevent detection of serious security violations that exist when events occur in a particular order. Profiles that anomaly detection techniques use also may be vulnerable to conscious manipulation by users. Consequently a knowledgeable perpetrator may train the thresholds of detection system adaptive profiles to accept aberrant behaviors as normal. Furthermore, statistical techniques that anomaly detection systems use require complicated mathematical calculations and, therefore, are usually computationally expensive.
Expert systems (also known as rule-based systems or production systems) have had some use in misuse detection, generally as a layer on top of anomaly detection systems for interpreting reports of anomalous behavior. Since the underlying model was anomaly detection, they have the same drawbacks of anomaly detection techniques.
Expert system approaches, in addition, are themselves inherently inefficient. S. Snapp, et al., "DIDS (Distributed Intrusion Detection System)" Proc. 14th Nat'l Computer Security Conf., Washington, D.C. (October 1991) describes one example of an expert system signature analysis model that detects misuse by looking for one specific event within a specific system context. In one study, this detection system was found to be two and four orders of magnitude slower than "hard-wired" techniques and much too slow for real-time operation. This also makes it impractical to use these systems to detect and report misuses of multiple associated processing systems through operation of a single misuse detection and reporting system.
Expert systems approaches are also not deterministic. Consequently, these rules are expressed in a declarative, non-procedural fashion. When rule changes occur, it is generally extremely difficult to predict how the new system will behave. This makes development and testing more complex and expensive. Moreover, expert system approaches are limited to the knowledge of the expert who programmed the rules into the system. However, an expert is only capable of programming the rules for behavior that the expert knows. Since there are often many different paths to a particular misuse, the expert will unable to create rules that represent all of these paths.
Consequently, there is a need for a method and system that provides an independent capability for detecting and reporting misuses and facilitating their subsequent investigation.
There is a need for a method and system for automatically recognizing intrusions and misuses of one or more data processing systems that minimizes the number of false positive misuse reports, eliminates the need for expert system programmers to enter knowledge database rules, and permits rapid processing of data from multiple systems using a single computer.